In many applications involving access to and control of a computer system through a workstation interface, the user moves a cursor around the associated workstation screen by physically moving a hand-held device called a "mouse" across a flat surface. The user triggers certain events by pressing a button on the mouse; this action is often referred to as "clicking" the mouse. In general, each mouse-clicked event is associated with one data object, such as a data file, out of the many objects displayed on the screen. The object selected is the one pointed at by the cursor.
Simple, mouse-driven interfaces have been exploited in numerous commercial systems. The mouse in these systems typically has a single button. The repertoire of mouse-clicked operations is somewhat small. These operations include, for example: (1) selecting a data object in preparation for performing an operation on that object; (2) opening a data object (e.g., editing a file); and (3) "dragging" an object to a new location on the screen. The key characteristic of this user interface is that it is easily learned, often without the need for extensive reading of manuals or formal training. However, this style of user interface has limitations: (1) there are very few functions immediately available within a single mouse click; and (2) as a result, routine, repeated operations are laborious to perform.
Other commercial systems, particularly those serving the artificial intelligence (AI) community, use more sophisticated mouse-driven interfaces. Typically, a mouse in these systems has multiple buttons, with each button having a different function. In some applications involving a mouse with more than one button, the user triggers certain events by pressing multiple buttons on the mouse simultaneously; this action may be referred to as "chording" the mouse. Single-button "clicks" (as described above) are degenerate cases of "chords"---- i.e., they are chords in which only a single button is pressed.
In these interfaces, the functions associated with the one or more buttons change depending upon the state of the computer system. Sometimes there is no information presented visually to the user concerning the change in function of the buttons. In other systems, the functions for each button are indicated by a legend line or "mouse line" which is typically located in a region of the screen disassociated from the cursor location. The user is often unaware of changes in function because the legend line has been unobtrusively rewritten.
Several systems modify the appearance of the cursor to indicate the different functional states of the system. Typical default cursor shapes are shown in FIGS. 1 and 2. These may be replaced by other shapes or icons representative of the current computer functionality. However, this leads potentially to the problem of conflict between two desirable cursor functionalities: the need to represent a computer operation through a displayed icon, and the need to select unambiguously a single data object as the target of that computer operation. As shown by the tiny circled cross-hairs (or "hot spots") which have been superimposed in FIGS. 3 and 4, the default cursor shapes of FIGS. 1 and 2 provide unambiguous pointer functionality. The same cannot be said of many operation icons that might be substituted for the default shapes. The result of this problem is that certain cursor shapes with high information content cannot be used because they are ambiguous pointers. Certain other cursor shapes with low information content are used repeatedly to represent more than one computer operation, because they make good pointers. This generally leads to a corresponding ambiguity as to which operation is currently active or available.
AI environments often increase the functionality of the cursor through the use of pop-up menus. One mouse button --typically the left button--is reserved as a "select" button. Other mouse buttons are used to signal the computer system to display a temporary menu (i.e., a pop-up menu) of possible operations. The mouse is then moved over the pop-up menu until it is positioned on the desired operation. The "select" button is used to invoke the desired operation. A succession or "cascade" of popup menus may be required to complete the selection of the desired operation. When the selection process is complete, all pop-up menus are removed, and the selected operation is performed.
While pop-up menus increase the power of the system, they also have limitations: (1) the available commands are hidden from the user until the pop-up menu is invoked; (2) as a result, the user must explore all possible pop-up menus to understand what functionality is available for the current state of the computer system; and (3) the user who is experienced in the system and who already knows the options of the pop-up menus is nonetheless forced to traverse those menus (i.e., there are no short-cuts obviating pop-up traversal available through the mouse and cursor).
These arrangements make the AI environments suboptimal for several classes of users. The hidden and changeable nature of the operations makes the AI environments difficult to use except by those well-versed in the particulars of a given system. The requirement of pop-up menu traversal imposes extra, unnecessary steps upon experienced users who no longer need the reminder/prompter function of the pop-up menus.
Today, system developers are designing more and more powerful systems and tools, and they are doing so within an AI-like framework. There is a need to have this technology utilized by users not educated in the subtleties of computer science or artificial intelligence. What is required is an interface arrangement and method that transforms display and operational processes into a less imposing, more user-friendly environment without sacrificing the AI-like capabilities. There is a concomitant need to retain the speed and power of the AI-like environment for experienced users.
An arrangement describing the trend towards the desired interface arrangement is disclosed in the paper "Corporate Identity for Iconic Interface Design: The Graphic Design Perspective," authored by A. Marcus and appearing in IEEE Computer Graphics and Applications vol. 4, issue 12, Dec. 1984, pp. 24-32. This paper shows several cursor shapes, each of which includes a frame into which an icon can be overlaid, and a pointer functionality which is used to select a data object as the target of the computer operation whose icon is displayed in the frame. FIG. 5 shows an empty frame; FIG. 6 shows a frame with a computer operation icon overlaid within it (FIGS. 5 and 6 are redrawn from the A. Marcus paper, at a 16.times.16 resolution for compatibility with the present invention, rather than the 64.times.64 resolution of the original paper). The paper shows two different frames, which are used within the same environment. One of these is a square frame with a pointer attached to it, as shown in FIG. 5. The other is simply a square (FIG. 7) whose upper left corner serves by convention as the implicit pointer of the cursor. Hot spots for these cursor shapes are shown in FIGS. 8 and 9.
This is an improvement, in that it reduces the cognitive load upon the user. The current computer operation which has been loaded into the cursor is iconically displayed as a constant reminder of that operation. The icon of the computer operation itself is not required to serve as a pointer, because it is overlaid within the frame of the cursor. The frame of the cursor provides the pointer functionality. However, the technique is limited to a single computer operation at a time: only one icon can be made visible within the frame at any moment. This limitation contributes to some of the problems described above. Some commands remain invisible to the naive user, hidden within pop-up menus which are activated by multiple mouse buttons. The use of two different pointer frames (i.e., FIG. 5 and FIG. 7)--one with an implicit pointer functionality--in the same user interface contributes to user confusion, the opposite of the desired goal.
A second aspect of the trend toward the desired interface arrangement is disclosed in the paper "Experiences with The Alternate Reality Kit--An Example of the Tension Between Literalism and Magic," authored by R. B. Smith and appearing in the Proceedings of CHI+GI (Toronto, April 5-9), ACM, New York, 1987, pp. 61-67. This paper describes an animated environment for creating interactive simulations. Users of the environment manipulate objects with a mouse-operated "hand" which enables the user to simulate the carrying or throwing of objects, the pressing of buttons, and the operating of sliding mechanisms on the workscreen. The hand can be made to pick up a tool and to carry that tool to an object which is its target, where the hand can then activate the tool by pressing a button which is displayed on the tool icon.
Replacement of the cursor with the hand icon and the provision of additional types of operations traditionally performed by the human hand imparts an added sense of dimensionality and user friendliness to the usual cursor control technique. However, this technique does have its shortcomings in that the functionality of the arrangement is described in terms of operations or activities performed by the human hand. Only one tool can be used at any particular moment. Tools in the Alternate Reality Kit environment are blunt rectangular objects rather than precise pointers, and are therefore incapable of directing their activity to a narrowly-defined area or data object.
Furthermore, under both the Alternate Reality Kit and the work by A. Marcus, the tool icon is continuously displayed. This has costs in terms of display complexity: For many-button or many-protocol mice, a large and detailed cursor shape must be moved about the display, obstructing the user's view of the data objects. The technique also has costs in terms of operational overhead: The continuous movement of large cursor displays requires non-trivial computing resources.